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Abstract. This short note aims at proving that the isolation problem is 
undecidable for probabilistic automata with only one probabilistic tran- 
sition. This problem is known to be undecidable for general probabilistic 
automata, without restriction on the number of probabilistic transitions. 
In this note, we develop a simulation technique that allows to simulate 
any probabilistic automaton with one having only one probabilistic tran- 
sition. 

1 Introduction 

Probabilistic automata. Rabin introduced probabilistic automata over fi- 
nite words as a natural and simple computation model [Rab63]. A probabilis- 
tic automaton can be thought as a non-deterministic automaton, where non- 
deterministic transitions are chosen according to a fixed probabilistic distribu- 
tion. Probabilistic automata drew attention and have been extensively studied 
(see [Buk80] for a survey). 

The isolation problem. However, on the algorithmic side, most of the results 
are undecidability results. The isolation problem asks, given some probability 
< A < 1, whether there exists words accepted with probability arbitrarily 
close to A. Bertoni showed that this problem is undecidable [Ber74,BMT77] . 
Contribution. In this note, we prove that the isolation problem is undecidable, 
even for probabilistic automata having only one probabilistic transition. To do 
this, we develop a simulation technique that allows to simulate any probabilistic 
automaton with one having only one probabilistic transition. 
Outline. Section 2 is devoted to definitions. In section 3, we develop a simulation 
technique, which allows to simulate any probabilistic automaton with one having 
only one probabilistic transition. Using this technique we show that the isolation 
problem is undecidable for this very restricted class of automata. 



2 Definitions 



Given a finite set of states Q, a probability distribution (distribution for short) 
over Q is a row vector 8 of size |Q| with rational entries in [0,1] such that 



J2 q eQ ^(s) ~ 1- We denote by 5 g the distribution such that S q (q') = 1 if q' = q 
and otherwise. A probabilistic transition matrix M is a square matrix of size 
\Q\ x \Q\, such that for a state s, M a (s, _) is a distribution over Q. 

Definition 1 (Probabilistic automaton). A probabilistic automaton is a tu- 
ple A = (Q, A, [M a ) a ^Ai <?o> F), where Q is a finite set of states, A is the finite 
input alphabet, (M a ) ag ^ are the probabilistic transition matrices, qo is the initial 
state and F is the set of accepting states. 

For each letter a <G A, M a (s, t) is the probability to go from state s to state 
t when reading letter a. A probabilistic transition is a couple (s, a) such that 
M a (s,t) £ {0, 1} for some t. 

A probabilistic automaton is said simple if for all a, for all states s and t, we 
have M a (s,t) G {0,f,l}. 

Given an initial distribution 5 and an input word w, we define 5 ■ w by 
induction on w: we have 5 ■ e = 6, then for a letter a in A, we have S ■ a = M a • 6 
and if w = v ■ a, then 8 ■ (v ■ a) = (6 ■ v) ■ a. 

We denote by P^(s T) the probability to reach the set T from state s 
when reading the word w, that is J^teri^ ' w )(^)- 

Definition 2 (Value and acceptance probability). The acceptance proba- 
bility of a word w € A* by A is F^(w) = Pa{°o ~^ F)- The value of A, denoted 
val(A), is the supremum acceptance probability: val(A) = sup.^g^* P^(to). 

3 Simulation with one probabilistic transition 

Wc first show how to simulate a probabilistic automaton with one having only 
one probabilistic transition, up to a regular language: 

Proposition 1. For any simple probabilistic automata A = (Q, A, (M a ) ae A, Qo-, F), 
there exists a simple probabilistic automaton B over a new alphabet B, with one 
probabilistic transition, and a morphism 1 : A* t— > B* such that: 

Vw e A*,F A (w) = ¥ B (w). 

The morphism C will not be onto, so this simulation works up to the regular 
language {w | w € A*}. We shall see that the automaton B will not be able to 
check that a word read belongs to this language, which makes this restriction 
unavoidable in this construction. 

Wc first give the intuitions behind the construction. Intuitively, while reading 
the word w, the probabilistic automaton A "throw parallel threads". A com- 
putation of A over w can be viewed as a tree, where probabilistic transitions 
correspond to branching nodes. 

On the figure, reading a from go or b from q\ leads dctcrministically to the 
next state. Reading b from q2 leads at random to r or to s, hence the corre- 
sponding node is branching. Our interpretation is that two parallel threads are 
thrown. Let us make two observations: 




Fig. 1. An example of a computation 



— threads are not synchronised: reading the fourth letter (an a), the first thread 
leads deterministicaUy to the next state, while the second thread randomizes; 

— threads are merged so there are at most n = |Q| parallel threads: whenever 
two threads synchronize to the same state q, they are merged. This happens 
in the figure after reading the fifth letter (b). 

The automaton B we construct will simulate the n threads from the begin- 
ning, and take care of the merging process each step. 

Proof. We denote by qt the states of A, i.e Q = {qo, . . . , q n -i}- The alphabet B 
is made of two new letters '*' and 'merge' plus, for each letter a <G A and state 
q<EQ, two new letters check(a,g) and apply(a,g), so that: 

B = {*, merge} U [J {check(a, q), apply(a, q)} 

a£A,q£Q 

We now define the automaton B. We duplicate each state q €E Q, and denote 
the fresh copy by q. Intuitively, q is a temporary state that will be merged at 
the next merging process. States in B are either a state from Q or its copy, or 
one of the three fresh states s*, sq and si. 

The initial state remains go as well as the set of final states remains F. 

The transitions of B are as follows: 

— for every letter a € A and state q £ Q, the new letter check(a, q) from state 
q leads deterministicaUy to state s* i.e M check / a ,q){<l) = s *> 

— the new letter * from state s* leads with probability half to sq and half to 
Si, i.e Al St (*) = is + 5S1 (this is the only probabilistic transition of B); 

— the new letter apply(a, q) from states Sq and si applies the transition function 
from q reading a: if the transition M a (q) is deterministic, i.e M a (q,r) = 1 
for some state r then M app i y ( a , q )(so) = f an d -^appiy(a,g) ( s i) = f, else the 
transition M a (q) is probabilistic i.e M a (q) — \r + |r' for some states r,r', 
then M apply(a ^)(s ) = r and M apply(ajg) (si) = r'; 

— the new letter merge activates the merging process: it consists in replacing 
q by q for all q € Q. 



applyO, q) 
so ► r 



check(a, q) 
q S* 



apply(a, q) 
si ► n 



Whenever a couple (letter, state) does not fall in the previous cases, it has no 
effect. The gadget simulating a transition is illustrated in the figure. 
Now we define the morphismC : A* i-> B* by its action on letters: 

a = check(a, qo) ■ * ■ apply(a, q<j) . . . check(a, q n -i) • * ■ apply(a, q n -i) • merge. 

The computation of A while reading w in A* is simulated by B on w, i.e we 
have: 

¥ A (w) = ¥ B (w) 

This completes the proof. □ 

Let us remark that B is indeed unable to check that a letter check(a, q) is 
actually followed by the corresponding apply (a, q): inbetween, it will go through 
s* and "forget" the state it was in. 

We now improve the above construction: we get rid of the regular external 
condition. To this end, we will use probabilistic automata whose transitions have 
probabilities 0, |, | or 1. This is no restriction, as stated in the following lemma: 

Lemma 1. For any simple probabilistic automata A = (Q, A, (M a ) a ^Aj QOi F)> 
there exists a probabilistic automaton B whose transitions have probabilities 0, 
|, | or 1, such that for all w in A*, we have: 

val(A) = val(B). 

Proof. We provide a construction to pick with probability half, using transitions 
with probability 0, |, | and 1. The construction is illustrated in the figure. 

2 
3 

«- ro 

o 
. o 

► n 

In this gadget, the only letter read is a fresh new letter (J. The idea is the 
following: to pick with probability half ro or n, we sequentially pick with proba- 
bility a third or two thirds. Whenever the two picks arc different, if the first was 






a third, then choose ro, else choose r\. This happens with probability half each. 
We easily see that P^(a • a\ ■ . . . ak-i) = sup p Pe(ao • tl p • 0,1 • tt p • • • a fc-i • tt p )- n 

Proposition 2. For any simple probabilistic automata A = (Q, A, (Af a ) ag ^, qq, F), 
there exists a simple probabilistic automaton B over a new alphabet B, with one 
probabilistic transition, such that: 

val{A) > A <=> val(B) > A. 

Thanks to the lemma, we assume that in A, transitions have probabilities 0, 
± 2 or 1 

We first deal with the case where A = 1. The new gadget used to simulate a 
transition is illustrated in the figure. 



check(a, q) 

q 



TO -< 

apply(a, q) 



apply(a, q) 

The automaton B reads words of the form u\ -finish -finish . . ., where 'finish' 
is a fresh new letter. The idea is to "skip" , or "delay" part of the computation 
of A: each time the automaton B reads a word Ui, it will be skipped with some 
probability. 

Simulating a transition works as follows: whenever in state s*, reading two 

times the letter '*' leads with probability half to Si, quarter to Sq and quarter 

to s. As before, from s and s±, we proceed with the simulation. However, in the 

last case, we "wait" for the next letter 'finish' that will restart from q . Thus each 

time a transition is simulated, the word being read is skipped with probability 
1 

I- 

Delaying part of the computation allows to multiply the number of threads. 
We will use the accepted threads to check the extra regular condition we had 
before. To this end, as soon as a simulated thread is accepted in B, it will go 
through an automaton (denoted C in the construction) that checks the extra 
regular condition. 

Proof. We keep the same notations. The alphabet B is made of three new letters: 
'*', 'merge' and 'finish' plus, for each letter a £ A and state q £ Q, two new 
letters chcck(a, q) and apply(a, q), so that: 




B = {*, merge, finish} U [J {check(a, q), apply (a, q)} 

aeA,q£Q 



We first define a syntactic automaton C. We define a morphismC : A* i— y B* 
by its action on letters: 

a = check(a, qo) ■*■*■ apply(a, qo) . . . chcck(a, q n -x) ■ * ■ * ■ apply(a, q n -i) ■ merge. 

Consider the regular language L = {w- finish | w £ A*}*, and C = (Qc,$c, s C,Fc) 
an automaton recognizing it. 

We now define the automaton B. We duplicate each state q £ Q, and denote 
the fresh copy by q. States in B are either a state from Q or its copy, a state 
from Qc or one of the four fresh states s*, so, *i an d wait. 

The initial state remains qo, and the set of final states is Fq. 

The transitions of B are as follows: 

— for every letter a £ A and state q £ Q, the new letter check(a, q) from state 
q leads deterministically to state s* i.e M c h ec u a!g \(q) = s*, 

— the new letter * from state s* leads with probability half to s* and half to 
so, i.e M St (*) = ia + + is (this is the only probabilistic transition of B): 

— any other letter from state s* leads deterministically to w, i.e M St (_) = wait; 

— the new letter * from state sq leads deterministically to si, i.e M So (*) = s\; 

— the new letter apply(a, q) from states sq and Si applies the transition function 
from q reading a: if the transition M a {q) is deterministic, i.e M a (q,r) = 1 
for some state r then M app i y ( a q ^ (sq) = f and M app i y ( , g )(si) = f, else the 
transition M a (q) is probabilistic i.e M a (q) = |r + |r' for some states r,r', 
then M apply(a ^)(s ) = f and M app i y ( aj9 )(si) = r'; 

— the new letter merge activates the merging process: it consists in replacing 
q by q for all q £ Q; 

— the new letter finish from state wait leads deterministically to qo] 

— the new letter finish from state q in F leads deterministically to sc; 

— the new letter finish from any other state is not defined (there is a determin- 
istic transition to a bottom non-accepting state). 

Transitions in C are not modified. Whenever a couple (letter, state) does not fall 
in the previous cases, it has no effect. 

We now show that this construction is correct. 

We first prove that for all w £ A* , there exists a sequence of words (u> p ) p >i 
such that P,a(w>) = sup p P,g(u;p). 

We have, for 5 a distribution over Q: 

3 1 
Sb{S,o) = -5^(6,0.) + -wait. 

It follows: 

Sb(S 7 w) = (jj S A (S,w) + l 
where k — \w\. Hence: 

5 B (q , w ■ finish) = f^j ¥ A (w) + 1 - ( ^ q . 



3 x fc 

4 



wait, 



The computation of A while reading w is simulated by B on w ■ finish. This 
implies that sup p Pg((u;-finish) p ) =F^(w), hence ifval(.4) = 1, then val(£>) = 1. 

Conversely, we prove that if val(£>) = 1, then val(_4) = 1. Let w a word 
read by B accepted with probability close to 1, we slice it as follows: w = 
u\ ■ finish • . . . • ■ finish, such that ui does not contain the letter finish. The 
key observation is that if k = 1, the word w is accepted with probability at 
most |. Hence we consider only the case k > 1. We assume without loss of 
generality that Va( u i) > (otherwise we delete u x ■ finish and proceed). In this 
thread has been thrown while reading u\ that reached sc , so the syntactic 
process started: it follows that Ui for i > 1 arc in the image of C This implies 
that the simulation is sound: from w we can recover a word in A* accepted with 
probability arbitrarily close to 1 by A. 

The case where A is any positive rational is handled similarly. We only need 
to ensure that the previous key observation still holds: a word of the form it-finish 
where u does not contain finish cannot be accepted with probability more than 
This is made possible by slightly modifying the simulation gadget, adding 
new intermediate states. 

This completes the proof. □ 

We conclude: 

Theorem 1. The isolation problem is undecidable for simple automata with one 
probabilistic transition. 
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